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Kostka functions associated to complex reflection groups 


Toshiaki Shoji 


Abstract. Kostka functions ^{t) associated to complex reflection groups are 
a generalization of Kostka polynomials, which are indexed by a pair A, fi of r- 
partitions of n (and by the sign It is expected that there exists a close 

relationship between those Kostka functions and the intersection cohomology as¬ 
sociated to the enhanced variety ^ of level r. In this paper, we study combina¬ 
torial properties of ^(t) based on the geometry of In paticular, we show 
that in the case where = (—,..., —(and for arbitrary A), ^(t) has a 

Lascoux-Schiitzenberger type combinatorial description. 


Introduction 

In 1981, Lusztig gave a geometric interpretation of Kostka polynomials in the 
following sense; let V be an n-dimensional vector space over an algebraically closed 
field, and put G = GL{V). Let be the set of partitions of n. Let be the 
unipotent class in G labelled by A G and K = Q;) the intersection co¬ 

homology associated to the closure of Let Kx,^{t) be the Kostka polynomial 
indexed by A,/i G and Kx,fj.{t) = Kx,fj,{t~^) the modified Kostka polynomial 

(see 1.1 for the definition n(/i)). Lusztig proved that 

(0.1) Kx,^,{t) = ^ 

i>0 

for a; G C where is the stalk at x of the 2i-th cohomology sheaf 

of K. (0.1) implies that Kx^^it) G Z>o[t]. 

Let ^n,r be the set of r-tuple of partitions A = (A*-^^,..., A*^'’^) such that 
1 |A0)| = n (we write |A0)| = m if A^ ^ In [SI], [S2], Kostka functions 

associated to complex reflections groups (depending on the signs -b, —) are 
introduced, which are apriori rational functions in t indexed by A, /a G ^n,r- In 
the case where r = 2 (in this case it is proved in [S2] that 

^ I^ case, Achar-Henderson [AH] proved that those (generalized) 
Kostka polynomials have a geometric interpretation in the following sense; under 
the previous notation, consider the variety = G x K on which G acts naturally. 
Put = Guni X V, where Guni is the set of unipotent elements in G. ^uni is a 
G-stable subset of and is isomorphic to the enhanced nilpotent cone introduced 
by [AH]. It is known by [AH], [T] that has finitely many G-orbits, which are 
naturally parametrized by ^n, 2 - They proved in [AH] that the modified Kostka 

polynomial ^ ^n, 2 ), defined in a similar way as in the original case, 
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2 


can be written as in (0.1) in terms of the intersection cohomology associated to the 
closnre of the G-orbit C ^ni- 

In the case where r = 2, the interaction of geometric properties and combinato¬ 
rial properties of Kostka polynomials was stndied in [LS]. In particular, it was proved 
that in the special case where /x = (—(and for arbitrary A G ^n, 2 ), 
has a combinatorial description analogous to Lascoux-Schiitzenberger theorem for 
the original Kostka polynomials ([M, III, (6.5)]). 

We now consider the variety ^ = G x for an integer r > 1, on which 

G acts diagonally, and let ^ni = Guni x be the G-stable subset of ^. The 
variety ^ is called the enhanced variety of level r. In [S4], the relationship between 
Kostka functions indexed by A,/x G J^n,r and the geometry of was 

studied. In contrast to the case where r = 1,2, ^ni has inhnitely many G-orbits if 
r > 3. A partition = 1 Iag.^„^ G-stable pieces X\ was constructed in 

[S3], and some formulas expressing the Kostka functions in terms of the intersection 
cohomology associated to the closure of Xa were obtained in [S4], though it is a 
partial generalization of the result of Achar-Henderson for the case r = 2. 

In this paper, we prove a formula (Theorem 2.6) which is a generalization of the 
formula in [AH, Theorem 4.5] (and also in [FGT (11)]) to arbitrary r. Combined 
this formula with the results in [S4], we extend some results in [LS] to arbitrary r. 
In particular, we show in the special case where fi = /i^’’^) G ^n,r (and for 

arbitrary A G ^n,r) that ^(t) has a Lasacoux-Schiitzenberger type combinatorial 
description. 


1. Review on Kostka functions 

1 . 1 . First we recall basic properties of Hall-Littlewood functions and Kostka 
polynomials in the original setting, following [M]. Let A = A{y) = 0„>o be the 
ring of symmetric functions over Z with respect to the variables y = (i/i, 1 / 2 ,...), 
where A^ denotes the free Z-module of symmetirc functions of degree n. We put 
Aq^ = Q (8)z A, Aq = Q ®z A'^. Let sx be the Schur function associated to A G ^n- 
Then {sa | A G gives a Z-baisis of A'^. Let p\ G A" be the power sum symmetric 
function associated to A G 


k 

p\= 

i=l 

where pm denotes the m-th power sum symmetric function for each integer m > 0. 
Then {px \ A G gives a Q-basis of Aq. For A = 2 "^^^...) G ^n, dehne an 

integer 2 :a by 

(lA.l) zx = lli^^m,\. 

i>l 

Following [M, I], we introduce a scalar product on Aq by {px,Pfj) = ^x^iZx- It is 
known that {sa} form an orthonormal basis of A. 



3 


Let Px{y,t) be the Hall-Littlewood function associated to a partition A. Then 
{Pa I A G gives a Z[t]-basis of A"’[t] = Z[t\ 0z where t is an indeterminate. 
Kostka polynomials Kx^^(t) G Z[t\ (A,/i G ^n) are dehned by the formula 

(1-1-2) sx{y) = Kx,^,{t)P^{y]t). 

Recall the dominance order A > /r in which is dehned by the condition 
EkA,> Ej=i hi sach f > 1. For each partition A = (Ai,..., A^), we dehne an 

integer n(A) by n(A) = E^=i(* “ l)Ai- R is known that Kx,fj,(t) = 0 unless X > y, 
and that Kx^^it) is a monic of degree n(/i) — n(A) if A > /i ([M, III, (6.5)]). Put 
Kx,fj,{t) = t Kx^^{t~^)- Then Kx^^{t) G Z[f], which we call the modihed Kostka 
polynomial. 

For A = (Ai,..., Afc) G with Xk > 0, we dehne zx{t) G Q(t) by 
(1.1.3) zx{t) = zxYlil - 

i>l 

where ;2 a is as in (1.1.1). Following [M, III], we introduce a scalar product on 
Rq(^) = Q(^) A hj {px,Pij,) = Zx{t)5x,^- Then Px{y]t) form an orthogonal basis 
of Alt] = Z\t\ A. In fact, they are characterized by the following two properties 
([M, III, (2.6) and (4.9)]); 

(1-1-4) Px{y; t) = sx{y) + ^ wx^,{t)Sf,{y) 

/i<A 


with WXfiit) G Z[t] , and 

(1.1.5) (Pa, P^) = 0 unless X = y. 


1.2. We hx a positive integer r. Let S = S(a;) ~ yl(a;*^^^) 0 • • • (8) be 

the ring of symmetric functions over Z with respect to variables x = ..., x^'^'^), 

where = {xi\x 2 \ ■ ■ ■)■ We denote it as S = 0 ^>qS”, similarly to the case 
of A. Let ^n,r be as in Introduction. For A G X^n,r, we dehne a Schur function 
SA(a;) G S” by 

(1.2.1) SA(a;) = SA(i)(a;(^)) ■ ■ •SA(r)(a;(^E 

Then (sa | A G ^n,r} gives a Z-basis of S". Let C be a primitive r-th root of unity 
in C. For an integer m > 1 and k such that 1 < k < r, put 

i=i 
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where denotes the m-th power sum symmetric function with respect to the 

variables For A G J^n,r, we dehne px{x) G Sq = S” C by 

r mfc 

( 1 - 2 . 2 ) px{x )= 

k=i j=i ^ 

where \ ■ ■ ■, Xml) with Xml > 0- Then {px \ A G gives a C-basis of 

For a partition X^^'^ as above, we dehne a function Zx(k){t) G C(f) by 

rrik 

zxwit)= n(i - 

i=i 

For A G X^n,r, we dehne an integer 2 ;^ by 2:a = 111=1 wher Zx(k) is as in 

(1.1.1). We now dehne a function zx(t) G C(t) by 

r 

(1.2.3) zx(t) = zxY\_^\w{'t)- 

k=l 

Let S[f] = Z[t\ S be the free Z[f]-module, and Sc(t) = C(f) 0z S be the C(f)- 
space. Then {px{x) \ A G X^n,r} gives a basis of S^(t). We dehne a sesquilinear 
form on Sc(t) by 


(1-2.4) {px,P,,)= Sx,,,zx{t). 

We express an r-partition A = (A^),..., A^)) as A^^^ = {X^i\ ■ ■ ■, Xm^) with a 
common m, by allowing zero on parts Xj’'\ and dehne a composition c(A) of n by 


c(a) = (a;'>,...,a'; 


,(r) 


A^) \h) \(i) \h)') 


We dehne a partial order A > /x on ^n,r by the condition c(A) > c(^), where > 
is the dominance order on the set of compositions of n dehned in a similar way as 
in the case of partitions. We hx a total order A >- /x on ^n,r compatible with the 
partial order A > /x. 

The following result was proved in Theorem 4.4 and Proposition 4.8 in [SI], 
combined with [S2, §3]. 

Proposition 1.3. For each A G ^n,r, there exist unique functions P^ix', t) G '^^{t) 
{depending on the signs +, — ) satisfying the following properties. 

(i) Px{x;t) can be written as 


Px{x]f) = SA(a;) + '^u^^^{t)s^{x) 


with Ux^^{t) G Q(t). 
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(ii) {Px, Pf^) = 0 unless X = fi. 

1.4. P± {x; t) are called Hall-Littlewood functions associated to A G 0^n,r- By 
Proposition 1.3, for e G {+, — {P{ \ X G 0^n,r} gives a Q(t)-basis for SQ(t). For 
A, ^ G we define functions Kx^{t) G Q(t) by 

(1.4.1) SA(a:) = 

Kxf^{t) are called Kostka functions associated to complex reflection groups since 
they are closely related to the complex reflection gronp S'„ k (Z/rZ)” (see [SI, 
Theorem 5,4]). For each A G ^n,r, by putting n(A) = n(AB)) + ■ • ■ + n(A‘^^^), we 
dehne an a-fnnction a(A) on ^n,r by 

(1.4.2) a(A) = r • n(A) + lA^'^l + 2|A(=')| + • • • + (r - 1)|A(’’)|. 

We dehne modifed Kostka functions P^^(t) by 

(1.4.3) K^Jt) = 

Remark 1.5. In the case where r = 1, Px{x;t) coincides with the original Hall- 
Littlewood fnnction given in 1.1. In the case where r = 2, it is proved by [S2, Prop. 
3.3] that Px{x]t) = Px{x]t) G S[f], hence Kx^it) = Kxfj_(t) G Z[t]. Moreover 
it is shown that P^^(t) G Z[t], which is a monic of degree a(/x) — a(A). Thus 
Kxf^{t) G Z\t]. As mentioned in Introdnction P^^(t) has a geometric interpretation, 
which imples that P\{p ^) independent of the choice of the total 

order -< on ^n,r- In the case where r > 3, it is not known whether Hall-Littlewood 
functions do not depend on the choice of the total order -<, whether A'^^(t) are 
polynomials in t. 


2. Enhanced variety of level r 

2.1. Let V be an n-dimensional vector space over an algebraic closure k of a 
hnite held F^, and G = GL(V) ~ GLn- Let B = TU be a Borel subgroup of G, T a 
maximal torus and U the nnipotent radical of B. Let IF = Ng{T)/T be the Weyl 
group of G, which is isomorphic to the symmetric group S'„. By hxing an integer 
r > 1, pnt ^ = Gx and S'uni = G^ni x where Guni is the set of nnipotent 
elements in G. The variety ^ is called the enhanced variety of level r. We consider 
the diagonal action of G on Pnt ^n,r = {m = (lui, ..., nir) G Z>g \'^mi = n}. 
For each m G we dehne integers Pi = Pi(m) by pj = rui + ■ ■ ■ + m* for 

i = 1,... ,r. Let (Mj)i<j<„ be the total hag in V whose stabilizer in G coincides 
with B. We dehne varieties 
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r—1 

^ = {{x,Y,gB) eGx X G/B \ g-^xg G B,g-\ e 

i=\ 

r—1 

^^=[jg{Bx\[M,,), 

geG i=l 

and the map TTm : t by {x, v, gB) i—)■ {x, v). We also define the varieties 

r—1 

’%n,um = {{x,Y,gB) G G^ni X V''~^ X G/B \ g~^xg G U,g~^w G 

i=l 

r—1 

[jg{Ux\[M,^), 

geG i=l 

and the map Tirn,i '■ ^xa,uni —t ^m,unh similarly. Note that in the case where 
m = (n, 0,..., 0), (resp. ^m,um) coincides with ^ (resp. ^ni)- In that case, 

we denote etc. by ^,n, etc. by omitting the symbol m. (Note: here we 

follow the notation in [S4], but, in part, it differs from [S3]. In [S3], our 
are denoted by for the consistency with the exotic case). 

2.2. In [S3, 5.3], a partition of ^uni into pieces X\ is defined 

^uni= n X;,, 

where Xx is a locally closed, smooth irreducible, G-stable subvariety of ^ni- H 
r = 1 or 2, Xx is a single G-orbit. However, if r > 3, Xx is in general a union of 
infinitely many G-orbits. 

For m G let = Smi x • • • x Smr be the Young subgroup of IF = Sn- For 
m G ■:Sn,r, we denote by ^(m) the set of A G ^n,r such that |Ab)| = m*. The (iso¬ 
morphism classes of) irreducible representations (over Q;) of are parametrized 

by ^(m). We denote by Vx an irreducible representation of corresponding to 

A, namley Vx = Vxm ® ® I4(^); where V)j denotes the irreducible representation 

of Sn corresponding to the partition /i of n. (Here we use the parametrization such 
that F(„) is the trivial representation of Sn)- The following results were proved in 
[S3]. 

Theorem 2.3 ([S3, Thm. 4.5]). Put dm = dim,^^. Then {7Tin)*Q,i[dm\ is a sem- 
simple perverse sheaf equipped with the aetion ofW-m, and is decomposed as 

(7r„.)*Q;[d„.] ~ 0 Vx®\C{:^m,.^x)[dml 

Ae^(m) 

where S^x is a simple local system on a certain open dense subvariety of STm- 
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Theorem 2.4 ([S3, Thm. 8.13, Thin. 7.12]). Put d'^ = dim uni- 

(i) is a semisimple perverse sheaf equipped with the aetion ofW^, 
and is decomposed as 

(7r™,i)*Q;Km] - 0 Vx(^lC(Xx,Qi)[dimXx]. 

Ae^(m) 

(ii) We have IC(.£m, - IC(Xa, Qz)[dimXA - d'^]. 

2.5. For a partition A, we denote by X the dual partition of A. For A = 
(Ad),..., Ad)) e ^(m), we dehne A* G ^{m) by A* = ((A^d)*,..., (Ad))*). Assume 

that A G V{m). We write (A^))* as < • • ■ < in the increasing order, 

where ii = A^*\ For each 1 < i < r, 1 < j < £*, we dehne an integer n{i,j) by 

n(i.j) = (|A<‘>I + ■ ■ ■ + |A“-‘>|) + M*’ + • ■ ■ + A"- 

Let Q = Qx be the stabilizer of the partial hag {Mn{ij)) in G, and Uq the unipotent 
radical of Q. In particular, Q stabilizes the subspaces Mp^. Let us dehne a variety 
by 


r—1 

Xx = {{x,v,gQ) G Guni X x G/Q\ g~^xg G UQ,g~^v G JjMp,}. 

i=l 

We dehne a map tta : Xx —)■ X^ni by {x, v, gQ) i—)■ {x, v). Then tta is a proper map. 
Since Xa ~ G {Uq x ]Q. Mp.), Xx is smooth and irreducible. It is known by [S3, 
Lemma 5.6] that dimXA = dimXA and that ImTTA coincides with Xa, the closure 
of Xa in .Cuni- 

For A,/i G iXn, let Kx^p = Kx,p{l) be the Kostka number. We have Kx^p = 0 
unless A > /X. For A = (A^),..., Ad)), /x = (/i^),... ,/id)) g ^(m), we dehne an 
integer Kx,p by 

Xx,p = Kx(l)^p(l)Kx(2)^p(2) ■ ■ ■Kx(r)^p(r). 

We dehne a partial order X > pt on iXn,r by the condition A*-*) > pd) for i = 1, ..., r. 
Hence A > /x implies that A, ^ G iX{in) for a commom m. We have Kx,p = 0 unless 
X > pi. Note that X> pi implies that pi^ ^ X*. We show the following theorem. In 
the case where r = 2, this result was proved by [AH, Thm. 4.5]. 

Theorem 2.6. Assume that X G iXn,r- Then (7rA)*Q;[dimXA] is a semisimple 
perverse sheaf on Xx, and is decomposed as 

(7rA)*Qi[dimXA] ~ 0 0 IC(X^, Q0[dimX^]. 

/x<A 


( 2 . 6 . 1 ) 



2.7. The rest of this section is devoted to the proof of Theorem 2.6. First we 
consider the case where r = 1. Actually, the result in this case is contained in [AH]. 
Their proof (for r = 2) depends on the result of Spaltenstein [Sp] concerning the 
“Springer hbre” (7rA)“^(2:) for 2 ; G Xa in the case r = 1. In the following, we give an 
alternate proof independent of [Sp] for the later use. Let Q be a parabolic subgroup 
of G containing B, M the Levi subgroup of Q containing T and Uq the unipotent 
radical of Q. (In this stage, this Q is independent of Q in 2.5.) Let Wq be the 
Weyl subgroup of W corresponding to Q. Let Greg be the set of regular semisimple 
elements in G, and put Treg = Greg Fl T. Consider the map tjj : Greg —t Greg, where 


Greg = {{x,gT) e Greg X G/T \g ^xg e Treg} 


and Ip : {x,gT) 1 —)■ x. Then is a hnite Galois covering with group W. We also 
consider a variety 

G^g = {(x^gM) G Greg X G/M \ g ^xg G Mreg}, 
where Mreg = Greg n M. The map tp is decomposed as 


%P : G 


b' 


reg 


'^reg 


-G G 


reg, 


where tp' : {x,gT) 1 —)■ {x,gM), ip” : {x,gM) 1 —)■ x. Here tp' is a hnite Galois cov¬ 
ering with group Wq. Now is a semisimple local system on Greg such that 

End('^*Qz) ~ Qi[lH], and is decomposed as 


(2.7.1) 

where = Hornby(p,'^*Qz) is a simple local system on Greg- We also have 

(2.7.2) 0 p'0.if;,, 

p'&W^ 

where is a simple local system on G^^. Hence 


(2.7.3) ~ ~ 0 

(2.7.3) gives a decompostion of with respect to the action of Wq. Comparing 
(2.7.1) and (2.7.3), we have 
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( 2 . 7 . 4 ) 0 

pew'^ 

where (p : p') is the multiplicity of p' in the restricted thg-module p. 
We consider the map tt : G —)■ G, where 


G = {{x,gB) eG X G/B \ g-^xg e 5} ~ G 5, 
and TT : {x,gB) i—)■ x. We also consider 


G« = {{x,gQ) e G X G/Q \ g-^xg G Q} ~ G x'^ g. 

The map tt is decomposed as 

TT : G G« G, 

where n' : {x,gB) i-G {x,gQ), n" : {x,gQ) i—)■ x. It is well-known ([LI]) that 


(2.7.5) TT.Qi ~ 0 p(8lC(G,^g. 

pew^ 

Let Bm = B n M he the Borel subgroup of M containing T. We consider the 
following commutative diagram 


Gx^ B G X (g X-® B) M x^" Bm 

(2.7.6) r 

Gx^Q^^ GxQ M, 

where under the identihcation G x^ B cx G x^ [Q x^ B), the maps p,p are dehned 
by the quotient by Q. The map g is a projection to the M-factor of Q, and q is the 
map induced from the projection Q x B ^ M x Bm- is dehned similarly to n 
replacing G by M. The map r is dehned by (p, h * x) (p, hxh~^). (We use the 
notation h* x E Q x^ B to denote the i?-orbit in g x i? containing {h,x).) Here 
all the squares are cartesian squares. Moreover, 

(a) p is a principal g-bundle. 

(b) g is a locally trivial hbration with hbre isomorphic to G x Uq. 

Thus as in [S4, (1.5.2)], for any M-equivariant simple pervere sheaf Ai on M, there 
exists a unique (up to isomorphism) simple perverse sheaf A 2 on G^ such that 
p*A 2 [a\ ~ q*Ai[b], where a = dimg and b = dimG -|- dimf/g. 
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By using the cartesian squares in (2.7.6), and by (2.7.2), we see that — 
IC(GQ,i7:Q/),and is decomposed as 

(2.7.7) <0,- 0 p'0IC(G«,i?;,). 

p'ew^ 

By comparing (2.7.4) and (2.7.7), we have 

(2.7.8) 7r:iC(G«,.if;,) ^ 0 0 IC(G, .if,). 

pew^ 

Note that if p = Va for A G ^n, we have 

(2.7.9) IC(G,.i^,)|G„„, ^IC(:^A,Qz)[dim^A-2z/G] 

by [BM], where uq = dim 77. Hence by restricting on Gum, we have 


(2.7.10) <10(05,^;)>gIIg.,„ 0 

Ae^„ 

2.8. Now assume that Wq ~ S', for a partition p, where we put S', = 
S',! X • • • X S',j, if p = (pi,..., pfc) G Take p' = £ the sign representation of Wq. 
We have 

(2.8.1) (Ha : e) = (Vx7 : WJ = 

where Iwq is the trivial representation of Wq. 

The restriction of the diagram (2.7.6) to the “unipotent parts” makes sense, and 
we have the commutative diagram 

Gx^U i - Gx^iQx^U) - >Mx^^Um 

( 2 . 8 . 2 ) 

G X« guni G X guni Muni, 

where Um is the unipotent radical of and guni, Muni are the set of unipotent 
elements in Q,M, respectively. pi,5'i have similar properties as (a), (b) in 2.7. 
We consider IC(M, .Sf^^) on M, where is the simple local system on M^g 
corresponding to e G Wq. Then by (2.7.6), we see that 

p* IC(G^, X) - <f IC(M, .^/^). 

By applying (2.7.9) to M, \C{M, ^ IC(^^, Qi)[dim - 2 z/a^], where 

is the orbit in Muni corresponding to e under the Springer correspondence, and vm 
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is defined similarly to vq- If is known that is the orbit {e} C Muni, where e is the 
identity element in M. Hence IC(M, coincides with Q/[—2 z/m] supported 

on {e}. It follows, by (2.8.2) 

(2.8.3) The restriction of IC(G'^,=Sf/) on G Quni coincides with i*Qj[—2//^^], 
where i : G Uq ^ G x^ Quni is the closed embedding. 

We deifne a map ttq : G x^ Uq ^ Guni hj g * x gxg~^. Put G^ = G x^ Uq. 

Proposition 2.9. Under the notation as above, 

(i) <'IC(GQ,.if;')[2^^G]|Guni - (ttq)*Q z[dimG?]. 

(ii) ITe have 

(7rQ)*Q«[dimGf] ~ ^ 0 IC((^a, Q«) [dim ^a]- 

/i<*A 

Proof. Note that 2ug — 2um = 2dimGQ = dimG)^. Thus by (2.8.3), 

(2.9.1) IC(G«,i?,')[2,.ollGx«Q.., - i,Q,[dimG?]. 

By applying the base change theorem to the cartesian square 


Gx'^gnni -1 GX^Q 



we obtain (i) from (2.9.1) since ttq = tt" o i. Then (ii) follows from (i) by using 
(2.7.10) and (2.8.1). □ 

2.10. Returning to the setting in 2.5, we consider the case where r is arbitrary. 
We £x m G ^n,r, and let P = P^ be the parabolic subgroup of G containing B 
which is the stabilizer of the partial flag (Mpji<j<r. Let L be the Levi subgroup 
of P containing T, and = B P L the Borel subgroup of L containing T. Let 
Gl be the unipotent radical of Bl. Put Mp. = MpjMp^_.^ for each i, under the 
convention M^^, = 0. Then L acts naturally on Mp^, and by applying the definition 
of TTm,! : ^ to L, we cau define 


L (Gi X 17*4,), 

i=\ 

r—1 r—1 

^ n®i>. 

gSL 2=1 2=1 

and the map ^ uni uni similarly. 



12 


Let Q = Qx he as in 2.5 for A G ^(m). Thus we have B C Q G P, and 
QL = QhiL is a parabolic subgroup of L containing Bl. We consider the following 
commutative diagram 


( 2 . 10 . 1 ) 


^ n V 

«-^m,uni ^ ^ ^ *^111 uni 


91 


f «-^m,uni 


/31 


<5513 , Pi fy ^ c^P,Q 11 . 


Pi 


uni G X 


Pl 


P 

m,uni 


91 


■> 


L 

m.uni 


^rr 


where, by putting Puni = L^niUp (the set of unipotent elements in P), 




p 

m.uni 


'^m.uni 


c^P 

'^m,uni 


'^mjUni 


c^P^Q 

'^m,uni 


Uj(t/xnj''4.)=p„„ixp[i\4, 

g^P i i 

G = Gx^ (P„ni X 

i 

Px»(c/xp[A4,), 

i 

G x« (g„ni X JjMpJ, 
i 

PX«(g„niX JjMpJ. 


is a similar variety as .^^uni dehned with respecto to {L,Ql), namely, 

^ X n^pj- 

i 

The maps are dehned as follows; under the identihcation — G {U x 

n. MpJ, q;(, a'l are the natural maps induced from the inclusions G x (P x ])([ Mpi) -)■ 
G X (guni X riMp.) -)■ G X (Puni X ]\Mp^). p'l Q * (x, v) H- {gxg~^,gw). qi 
is dehned by {g,x,v) i-G- (T, v), where a; —)■ T, v i—)■ v are natural maps P —)■ 

L, Hi Mpi -p- rii Mp.. qi is the composite of the projection G x .^^uni '^^uni 
and the map .^^uni '^muni induced from the projection P x (P x Yl^pi) 
p X (Pi X n Mp.). qi is dehned similarly by using the map =^^’uni 

induced from the projection P x (guni x Y\^pi) —t P x ((g^juni x Y\^pi)- Vi 
is the quotient by P. pi and pi are also quotient by P under the identihcations 

,^^uni — G x^ '^^,uni) "^^uni — G x^ '^m,’uni- 1^1 dehned similarly to aj and 
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f3" is defined similarly to 7i'(. r[ is the natural map induced from the injection 
P X {U X Yl^pi) —t -P X (Quni X ^1 is natural map induced from 

the map P x^ {Quni x ipM„) ^ a.i X n Mp., g*{x,v) ^ {gxg \gv). 

Put vrj = a” o aj : S'm,uni —t ^muni- W^e have /3'/ o /3[ = vr^ and the diagram 

(2.10.1) is the rehnement of the diagram (6.3.2) in [S4] (see also the diagram (1.5.1) 
in [S4]). In particular, the map pi is a principal P-bundle, and the map gi is a 
locally trivial fibration with hbre isomorphic to G x Up x YViZi Mp^. Moreover, all 
the squares appearing in (2.10.1) are caetesian squares. Hence the diagram (2.10.1) 
satishes similar properties as in the diagram (2.8.2). 

Note that L ~ Gi x ■ ■ • x with Gi = GL{Mp^). Then Qp can be written as 
Ql — Qi X ■ ■ ■ X Qr, where Qi is a parabloic subgroup of G*. We have 

r 

n(G.)u.i X V, 

i=l 

r 

■Cfs =x n(Gf •)«.,. X c 

i=l 

r 

i=l 

where (Gj)uni, (G?*)uni, etc. denote the unipotent parts of G*, G^% etc. as in (2.8.2). 
The maps are induced from the maps (Gj)uni —t (G^*), (Gf*)uni —t (Gj)uni, 

and those maps coincide with the maps tt', tt" in 2.7 defined with respect to Gj. Note 
that Wq. ~ >S'(A(0)t for each i by the construction oi Q = Q\ in 2.5. Put 

£? = G 'X(UQ y = L x0‘ (Uq, X Vp,). 

and let iq : e-f be the closed embeddings. Let 

ttql : —)■ be the restriction of 13". Let ~ x ■ ■ ■ x be the 

L-orbit in where is the Gj-oribt in (Gj)uni x Mp^ of type 0). Note 

that if we denote by the Gj-orbit in (Gj)uni of type /ib), we have IC(^^(i), Q;) ~ 
IC(^^(i),Qi) Kl Q; (the latter term Q; denotes the constatn sheaf on MpJ. Hence 
the decompostion of tTq^ into simple components is described by considering the 
factors IC((^^(i), Q;). In particular, by Proposition 2.9, we have 

(2.10.2) J.Q,[dim ~ 0 Qp' "' 0 IC(W^, Q,)|dim 

/x<A 


By using the diagram (2.10.1), we see that 

?L(fQz,)*Q;[dim ~ ^(fQ)*Qi[dimXA]. 
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It follows, again by using the diagram (2.10.1), we have 
(2.10.3) (a")*(^Q)*Q^[dimXA] ~ 0 ® 5^, 


where is the simple perverse sheaf on characterized by the property that 

p\B^[a'\ ~ ql IC(W^, Q,)[6' + dim dij] 

with a' = dim P, b' = dim G + dim Up + dim ni=i ^Pi- 
On the other hand, by Proposition 1.6 in [S4], we have 

where n” : (-P x ])([j Mp^) — )■ is an analogous map to tt", and is 

a simple perverse sheaf on such that the restriction of on coincides 

with Bp,, up to shift. Thus by Theorem 2.4 (ii), we have 


(2.10.4) {0*Bp ~ IC(X^,Q,)[dimX^]. 

Since ti\ = tt" o a” o iq, by applying (tt")* on both sides of (2.10.3), we obtain the 
formula (2.6.1). This completes the proof of Theorem 2.6. 


3. G'^-invariant functions on the enhanced variety 
AND KOSTKA functions 

3.1. We now assume that G and V are dehned over F^, and let F : G —)■ 
G,F : V —)■ F be the corresponding Frobenius maps. Assume that B and T are 
F-stable. Then Xx and Xx have natrual F^-structures, and the map nx ■ Xx —)■ Xx 
is F-equivariant. Thus one can dehne a canonical isomorphsim ip : F*Kx U^Kx for 
Kx = (ta)*Qz. By using the decomposition in Theorem 2.6, p can be written as 
p = cp^jL® p^i, where is the identity map on and Pp : F*Lp Lp is 

the isomorphism induced from p for Lp = IC(X^, Q;). (Note that dim—dim 
is even if < A by [S4, Prop. 4.3], so the degree shift is negligible). We also 
consider the natural isomorphism <pp : F*Lp ^ Lp induced from the Fg-strucutre of 
Xp. By using a similar argument as in [S4, (6.1.1)], we see that 

(3.1.1) pp = q'^-<t>p, 

where dp = n{pL). We consider the characteristic function xl^ of Lp with respect 

_ jp 

to 0^, which is a G^-invariant function on X^. 
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3.2. Take E and assume that v G S^{va). For each = (x, v) G 

with V = (ui,..., Vr-i)i we dehne a variety by 


%,z = {(Wp^) : x-stable flag | Vi G hFp, (1 < i < r - 1), 

(3.2.1) 

(1 < * < ^^)}- 

If G Xj^, the variety is defined over Fg. Put gu,z{(!) = \‘^S,z\- Kx^pit) be 
the modified Kostka polynomial indexed by partitions A,/i. The following result is 
a generalization of Proposiition 5.8 in [AH]. 

Proposition 3.3. Assume that A, /x G t^n,r- For each z G Xj^, we have 


XLx{z) = q 

u<\ 


Proof. Let XKx,^ b® fb® characteristic function of Kx with respect to (p. By Theorem 
2.6 together with (3.1.1), we have 


(3.3.1) = 

On the other hand, by the Grothendieck’s fixed point formula, we have ~ 

l'^A^(^)^l ^ ^ Then if 2 : = [x, v) G Xjf, 

(3.3.2) |7r-^(^)^| = \^u,z\YlWxl)ixif\, 


where tixO) ■ ^xo) —t ^xO) is a similar map as tta applied to the case r = 1, by 
replacing G by G* = GL{Mp.), and Xi = x\jj^ has Jordan type z/hb It is known 

by [LI] that (a:*) = K^(i)^^(i){q) for a partition of m*. It follows, by 

applying (3.3.1) to the case where r = 1, and by the Grothendieck’s fixed point 
formula, we have 

I^A(*)(^*) I ~ ^ ^ X^(i)t^xO)tK^(i)(^q). 

^0)<x0) 


Then (3.3.2) implies that 


9u,z{q)^F:^txK^m^^(i){q)---K^ir)^^ir){q). 


(3.3.3) Xkx,‘p 
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Since is a unitriangular matrix with respect to the partial order ^ ^ A, 

by comparing (3.3.1) and (3.3.3), we obtain the required formula. □ 

Remark 3.4. In general, consists of inhnitely many G-orbits. Hence the value 
gu,z{(l) may depend on the choice of z G Xj^. However, if is a single G-orbit, 
then Xj^ is also a single G^-orbit, and gu,z{(l) is constant for 2 : G Xj^, in which case, 
we denote gu,z{,g) by g^{q). In what follows, we show in some special cases that 
there exists a polynomial 5 '')'(t) G Z[t] such that 5 '')'(g) coincides with the value at 
t = qof gi^it). 

3.5. We consider the special case where G .^(m') is such that m' = 0 for 
i = 1,... ,r — 2. In this case, consists of a single G-orbit. In particular, for 
A G dime^*IC(XA, Qi) does not depend on the chocie of 2: G X^. We dehne 

a polynomial ICi(^(t) G Z[t] by 

= E 

i>0 

The following result was proved in [S4]. 

Proposition 3.6 ([S4, Prop. 6.8]). Let A,/x G cind assume that is as in 

3.5. 

(i) Assume that z G X^. Then ^*IC(Xa,Qz) = 0 if i is odd, and the eigen¬ 
values of fx on J^^AC{XQi) are q\ In particular, XLxi.^) = iC^)(^^(g). 

(ii) = 

As a corollary, we have the following result, which is a generalization of [AH, 
Prop. 5.8] (see also [LS, Prop. 3.2]). 

Corollary 3.7. Assume that /x is as in 3.5. 

(i) There exists a polynomial g^fif) G Z[t] such that g\f;{q) coincides with the 
value at t = q of g^{t). 

(ii) We have 

(3,7,1) Kipt) = Am,.(.)(*’■) ■ ■ - Am 


Proof. By Proposition 3.6 (i) and Proposition 3.3, we have 


(3.7.2) ICJ„(«) = E 9?(i)A'ai.),.w(9) ■ • • 

i/<X 


By hxing /x, we consider two sets of functions {IC;^^(g) | A G I^n,r} and {g\f;{q) \ 

v G t^n,r}- If we notice that K^o) ,,(i)(g) ■ ■ -Xy.) ,,M(g) = q^^^l for 1 / = A, (3.7.2) 
shows that the transition matrix between those two sets is unitriangular. Hence 
gfi{q) is determined from IC)(^(g), and a similar formula makes sense if we replace 
q by t. This implies (i). (ii) now follows from (3.7.2) by replacing g by f. □ 
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3.8. In what follows, we assume that is of the form ^ = (—,..., with 
^ G In this case, gj^{t) coincides with the polynomial obtained 

from G^(i) (r)(o) discussed in [M, II, 2], On the other hand, we dehne a polynomial 

by 

(3.8.1) P^(i){y;t)---P^(r){y;t) = 

In the case where r = 2, ^( 2 ) {t) coincides with the Hall polynomial, and a simple 

formula relating it with ^( 2 )(^) is konwn ([M, III (3.6)]). In the general case, we 

also have a formula 


(3.8.2) 






The proof is easily reduced to [M, III (3.6)]. 

For partitions A, z/d)^... ^ we dehne an integer 


(1). u(r) by 


-5^(1) • • • s„(r) — ^ C^(l),...,!yM'5A- 
A 

In the case where r = 2, c^(i) ( 2 ) coincides with the Littlewood-Richardson coeffi¬ 
cient. 

For A G J^n,r, put 

(3.8.3) 6(A) = a(A) - r • n(A) = |Ad)| + 2|Ad)| + ... + (r - l)|Ad)|. 

The following lemma is a generalization of [LS, Lemma 3.4]. 

Lemma 3.9. Let A, /x G t^n,r, and assume that ^ = (—,..., Then we have 


(3.9.2) Kljt) = ««'•)-''<') 5^ c;„,. ,„K,An- 

■n&^n 

Proof. The formula (3.7.1) can be rewritten as 

(3.9.3) 

. (f ) ■ ■ ■ (f ). 

Substituting (3.8.2) into (3.9.3), we obtain (3.9.1). Next we show (3.9.2). One can 
write as 
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Hence 


!y(») 


(3.9.4) 

Sa(i) (y)--- Sam (y) = {t)--- ^am,^m {t)P^m {y, t) ■ ■ ■ P^m {y, t) 


On the other hand, 


(3.9.5) S;,(l) (!/)■•• S;,M(y) = ^ c^(i),...,amS^(i/) 

= *^ 1 ( 1 ),...,am ^v,ii^)Piiy'i'^)- 

By comparing (3.9.4) and (3.9.5), we have an equality for each ^ G J^n, 


Combining this with (3.9.1), we obtain (3.9.2). The lemma is proved. □ 


3.10. Let f}' = \' — 6*', Tj" = \" — 6” be skew diagrams, where 9' C A', 9" C X" 
are partitions. We dehne a new skew diagram t]' * rj" = X — 9 as follows; write the 
partitions A', A" as A' = (A'^,..., A^,), A" = (A'/,..., X'^,,) with A(,, > 0, A'^,, > 0. Put 
a = A". We dehne a partition A = (Ai,..., Xk'+k") by 


A. 


A' + a 
A" 

^i-k' 


for 1 < i < k', 

ioi k' + 1 <i <k' + k”. 


Write partitions 6*', 9” as 9' = (6*(,..., 6'(,,), 9” = (6*",..., 6*^'„) with 6*(,, > 0, 6^^',, > 0. 
We dehne a partition 9 = (6*i,..., 6*fe/+fc»), in a similar way as above, by 


^ I 6^' + a for 1 < i < /c', 

forP + l<i<P + P'. 

We have 9 G X, and the skew diagram y' * y" = X — 9 can be dehned. 

For A, /i G J^n, let SST{X, y) be the set of semistandard tableaux of shape A and 
weight y. Let A G ^n,r- An r-tuple T = (T^^\ ... is called a semistandard 

tableau of shape A if is a semistandard tableau of shape A^*^ with respect to 
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the letters ,n}. We denote by SST{X) the set of semistandard tableaux of 

shape A. For A E ^n,r, let A be the skew diagram * A*^^^ * • ■ ■ * A*-^^. Then 
T E S'S'T(A) is regarded as a usual semistandard tableau T associated to the skew 
diagram A. Assume n E We say that T G SST{X) has weight n if the 

corresponidng tableau T has shape A and weight n. We denote by SST{X,7i) the 
set of semistandard tableaux of shape A and weight n. 

3.11. In [M, I, (9.4)], a bijective map 0 

(3.11.1) 0 : SST(X, tt) ^ ]J {SST\X, u) X SST{u, tt)) 

was constructed, where SST^{X, u) is the set of tableau T such that the associated 
word w(T) is a lattice permutation (see [M, I, 9] for the dehnition). Under the 
identihcation SST{X,tt) ~ S'S'T(A, tt), the subset SST^{X,iy) of SST{X,iy) is also 
dehned. Then we can regard 6> as a bijection with respect to the set SST{X,n) 
(and SST%X, u)). 

In the case where r = 2, it is shown in [LS, Cor. 3.9] that |S'S'T°(A, z/)| coincides 
with the Littlewood-Richardson coefficient ^( 2 ). A similar argument can be 
applied also to the general case, and we have 

Corollary 3.12. Assume that X E h^n,r) ^ Then we have 

3.13. For a semistandard tableau S', the charge c(S') is dehned as in [M, 
III, 6]. It is known that Lascoux-Schiitzenberger Theorem ([M, III, (6.5)]) gives a 
combinatorial description of Koskta polynomials K\^^{t) in terms of sesmistandard 
tableaux. 


(3.13.1) iFA,^(f) = t<^\ 

sessT{\,^l) 


In the case where r = 2, a similar formula was proved for Kx,fj.{t) in [LS, Thm. 
3.12], in the special case where = (—,ju"). Here we consider K\^^j_{t) for general r. 
Assume that A G h^n,r and ^ G For T G SST{X,^), we write 0{T) = {D,S) 
with S G SST{u,^) for some u. we dehne a charge c(T) of T by c(T) = c(S'). We 
have the following theorem. Note that the proof is quite similar to [LS]. 

Theorem 3.14. Let A, /x G o,nd assume that /x = (—,..., Then 


K^Jt) = 


r-'i’'’. 

tgsst{\,^) 


Proof. We dehne a map T : SST{X,f) —)■ S'S'T(z/, ,^) by T i-)- S', where 

0{T) = {D,S). Then by Corollary 3.12, for each A G SST{i',^), the set 'F“^(S') 
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has the cardinality and by dehnition, any T G \1/ ^(S*) has the charge 

c(T) = c(^). Hence 

E = E E <<■>. 

TeSST{\i) SeSSTiy,^) 

The last equality follows from (3.13.1). The theorem now follows from (3.9.2). □ 

Corollary 3.15. Under the assumption of Theorem 3.If, we have 

Kl^^{l) = \SST{\,i)\. 


3.16. In the rest of this section, we shall give an alternate description of 
the polynomial g^{t) in the case where /x = For v G put 

R^{x;t) = Then {P^, \ v G t^n,r} gives a basis of 

S"[f]. We dehne funtions h^{t) G Q(t) by the condition that 


(3.16.1) P^{x;t) = hf{t)P^{x;t). 

The following formula is a generalization of Proposition 4.2 in [LS]. 
Proposition 3.17. Assume that fx = Then 

hfit) = 

Proof. The proof is quite similar to that of [LS, Prop. 4.2]. For A G J^n,r, we have 


SA(a;) = Sa(i) (a;^^^) • • • (x^’’^) 

r 

= nE*V>.-‘'>(‘’’)Gc.)G®;r) 

*=1 !a(») 

= E (E^Am,.(.iG)---AAH,„M(«')ftJ(«))p;G;(). 

MS^n.r ^ ^ 

Since SA(a;) = comparing the coefficients of P~{x]t), 

we have 
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(3.1T.1) *!„(«)= Z .(.)(«’■) ■■■/iV) 

Now assume that /x = If we notice that ^ 0 only 

when (3.9.3) implies that 

(3.17.2) Kljf)= Y. 

Since {Kx(i)^um{f) ■ ■ ■ K)^{r) is a unitriangular matrix, the proposition 

follows by comparing (3.17.1) and (3.17.2). □ 
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